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(54) Gigabit ethernet interface to synchronous network (Sonet) ring 



(57) A nnethod and apparatus for routing data pack- 
ets via a synchronous optical (SONET) network from de- 
vices connected to a local area network to devices con- 
nected to other local area networks. At each transport 
node in the SONET network the data packets are un- 
loaded from SONET payioads and requeued for trans- 
mission on an output path. This allows for easy resolu- 
tion of contention between devices transmitting at the 
same time and avoids pre-assigning SONET channels. 
Source and destination address 

Information from the data packets can be used to 
route the data packets through the SONET network. A 
time-lo-lrve indicator can be used to find the shortest 
path to a destination address in a SONET ring network 
or any bi-directional ring network. 
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Description 

FIELD OF INVENTION 

[0001] This invention is concerned with an apparatus 
and method by which packetized data from a local area 
network can be transmitted over a synchronous optical 
(SONET) network. This invention is also concerned with 
a method tor selecting the shortest path to a destination 
address in a bi-directional ring network. 

BACKGROUND OF THE INVENTION 

[0002] Local area networks, or LANS, are used to 
transmit packetized data. One type of LAN is known as 
the Ethernet which conforms to a family of IEEE stand- 
ards, IEEE 802.3. Although Ethernet LANs have been 
produced for over 20 years, recent technological ad- 
vances have increased the speed at which devices con- 
nected to the ethernet LAN can transmit and receive da- 
ta. In IEEE draft standard 802.3z an Ethernet LAN ca- 
pable of gigabit per second transmission rates is pro- 
posed. However, one limitation of current ethernet LANs 
is that they must limit the distance between devices to 
about 2 km, in order to allow their carrier-sense multiple- 
access collision-detection (CSMA/CD) contention man- 
agement systems to function effectively. A description 
of local area networks is found in the textbook Basic Da- 
ta Communications: A Comprehensive Overview by W. 
J. Bey da. 

[0003] Data transmitted over a LAN is packetized or 
subdivided into a stream of segments of varying length. 
Packet flow is often described as "bursty". Packets may 
flow intensely between two devices connected to the 
LAN and then flow may suddenly drop to zero. When 
packets flow intensely from a number of different devic- 
es simultaneously, contention problems may arise and 
it is necessary to share the bandwidth on the LAN be- 
tween contending devices. 

[0004] It would be desirable for devices connected to 
an ethernet LAN to be able to transmit data to or receive 
data from devices connected to separate ethernet 
LANS. In this regard, it is known to use expensive spe- 
cial purpose wide area network (WAN) routers inter- 
posed between LANs. 

SUMMARY OF THE INVENTION 

[0005] It would be desirable to provide a method and 
apparatus by which devices connected to a local area 
network can send packetized data through a SONET 
network to devices connected to other local area net- 
works. 

[0006] It would also be desirable to allow the capacity 
on SONET payloads between specified transport nodes 
to remain unasslgned if the capacity is not needed for 
carrying data packets between those transport nodes. 
[0007] It would also be desirable to rationally share 



bandwidth dynamically on a SONET network between 
contending packetized devices. 

[0008] It would also be desirable to provide a method 
to find the shortest route to a destination address on a 

5 bi-directional ring network. 

[0009] In one aspect there is provided a method for 
routing data packets at a transport node of a SONET 
network comprising the steps of: (a) receiving SONET 
payloads having data packets at said transport node; 

10 (b) removing said data packets from said SONET pay- 
loads; (c) tor each of said data packets: (i) determining 
a destination address; (ii) selecting an output path 
based on said destination address: and (iii) queuing said 
each data packet on a queue associated with said se- 

16 lected output path; (d) selecting data packets from a 
queue associated with a given output path; and (e) 
transmitting said selected data packets onto said given 
output path. 

The output path on said SONET network may comprise 
20 an optical fibre. 

The output path may comprise an input to a local area 
network. 

The method may further comprise the steps of: (a) for 
each data packet: (i) associating said each data packet 

25 with an identifier of an input path from which said each 
data packet was received; (ii) determining if an entry ex- 
ists in a table associated with said transport node match- 
ing a source address of said each data packet and, if 
said entry does not exist, creating an entry on an asso- 

30 ciated table containing (i) a source address of said each 
data packet, and (ii) said identifier. 
In a further aspect, the present invention provides a 
method for routing data packets in a SONET network 
comprising the steps of: (a) receiving a data packet at 

35 a transport node ; (b) determining a destination address 
for said data packet; (c) selecting an output path based 
on said destination address; (d) queuing said data pack- 
et on a queue associated with said selected output path; 
(e) selecting a data packet from said queue; and (f) 

40 transmitting said selected data packet onto said select- 
ed output path. The data packet may be received from 
a data packet generating device. The selected data 
packet may be included within one or more SONET pay- 
loads. 

45 In another aspect, the present invention provides a 
method for routing data packets at a transport node in 
a SONET network, said transport node capable of re- 
ceiving packets from and transmitting packets to first, 
second and third directions, said second and third direc- 

50 tions transmitting to or receiving from said SONET net- 
work, said first direction for transmitting to or receiving 
from a data packet generating device, said method com- 
prising the steps of: (a) receiving a data packet from one 
of the directions; (b) removing said data packet from one 

55 or more SONET payloads when said data packet is re- 
ceived from the second or third direction; (c) determining 
a destination address for said data packet; (d) selecting 
an output path based on said destination address; (e) 
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queuing said data packet on a queue asscxiated with 
said selected output path; (t) selecting a data packet 
from said queue: (g) including said data packet in one 
or more SONET payloads where said output path is in 
the second c :hird direction; and, (h) transmitting said 
selected data packet on said selected output path. For 
instance, said first, second and third directions are re- 
spectively north, west and east. 

BRIEF DESCRIPTION OF THE FIGURES 

[001 0] The present invention will be better understood 
with reference to the detailed description in which: 

Figure 1 is a schematic drawing of a known com- 
munications system: 

Figure 2 is a schematic drawing of a communica- 
tions system in accordance with one embodiment 
of the present invention. 

Figure 3 is a schematic drawing of a portion of a 
transport node of the system of Figure 2; and 

Figure 4 is a schematic drawing of a communica- 
tions system in accordance with another embodi- 
ment of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0011] SONET or "synchronous optical network" is a 
standard for synchronous digital fibre-optic networks. 
An equivalent international standard, the Synchronous 
Digital Hierarchy (SDH) has been adapted by the inter- 
national Telecommunications Union (ITU), formerly the 
International Telegraph and Telephone Consultative 
Committee (CCITT). These standards define standard 
rates, frame formats, and optical signal parameters for 
signals passed on fibre-optic cables between transport 
nodes. SONET networks are described in Digital Te- 
lephony . Second Edition, by J. Bellamy. 
[0012] In SONET, a basic building block is the syn- 
chronous transport signal level 1 (STS-1) frame. Each 
frame is a 9 row by 90 column byte matrix which is se- 
rially transmitted such that the byte from row one. col- 
umn one. is transmitted first, followed by the byte from 
row one, column two, and so on, from left to right and 
from top to bottom. An STS frame is transmitted every 
1 25 micro-seconds or at 8 kHz. This 8 kHz transmission 
of frames is known as a channel. A number of channels 
can be transmitted on a single optical fibre. If N channels 
are transmitted this is known as STS-N. The N channels 
are interleaved so that every 125 microseconds the 
frames from all N channels are transmitted. The STS-1 
format results in a serial transmission rate of 51 .84 meg- 
abits per second. The frame consists of two types of in- 
formation, overhead and payload. Overhead comprises 
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data required to route and control the frame. There are 
three types of SONET overhead: path, line and section. 
Payload is the customer data routed on the SONET net- 
work. 

5 [0013] The payload, which has a maximum capacity 
of 49.536 Mbit/s on each STS-1 channel, could consist 
of a DS-3 signal at 44.736 Mbit/s or 28 DS-ls, each at 
1 .544 Mblt/s, a number of data channels, or other map- 
pings. Several STS-1 frames can be combined for trans- 

10 port of payloads that have bandwidth requirements 
greater than 50 Mbit/s. This process of combining STS- 
1 payload envelopes is referred to as concatenation and 
the concatenation of N STS-1 frames is denoted STS- 
Nc. Concatenation increases the ratio of payload to 

15 overhead because redundant overhead information in 
the concatenated frames is replaced by additional pay- 
load. A single optical fibre can carry many STS chan- 
nels. For example in STS-48, 48 channels are transmit- 
ted in the same optical fibre. When an STS-N signal is 

20 transmitted over a SONET network, the resulting optical 
signal is called "optical carrier N". or OC-N. 
[0014] SONET networks comprise transport nodes 
and cross-connects. Transport nodes are typically con- 
nected by optical fibres in a ring topology, or a mesh 

25 topology. Data is loaded into a SONET payload at an ^ 
originating transport node and transmitted from that 
transport node to adjacent transport nodes through one 
or more optical fibres. The payload observes SONET 
framing and channel structures and formats. 

30 [0015] It would be desirable to connect LANS by a 
SONET network. This would provide wide-area network 
(WAN) performance using only LAN and SONET com- 
ponents. However, current SONET networks are poorly 
designed to efficiently carry such data traffic. More par- 

35 ticularly, in a current SONET network, when an STS . 
payload (i.e. the payload part of an STS frame) or a part 
of an STS payload or a number of concatenated STS 
payloads are used to transmit information between any 
two communicating devices, such as transport nodes. 

40 routers, servers, cross-connects, or switches, on the 
network, those payloads (or that bandwidth) are p reas- 
signed to those two communicating devices and are un- 
available for data transmission between any other com- 
municating devices, such as transport nodes, routers, 

45 servers, cross-connects, or switches, on the network. In 
other words if an STS payload is assigned to transmit 
data from a first communicating device to a second com- 
municating device on a SONET network, no other data 
can be transmitted in that payload between other com- 

50 municating devices on the network, even at times when 
there is no data that is being exchanged between the 
first and second communicating devices. In existing 
SONET networks, STS payloads are reserved for trans- 
mission of data between particular communicating de- 

55 vices for long periods of time. The payloads are re- 
served because in existing systems it is resource-inten- 
sive to re-allocate a channel. This makes existing SON- 
ET networks unsuitable for bursty, packetized data traf- 
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[0016] A known SONET system is shown in Figure 1 
In communications network 400, LAN's 402a, 402b. 
402c, 402d can communicate with each other via fibre 
optical pairs 403, but oniy in a cumbersome fashion. 
Each LAN is connected to a proximate transport node 
404a, 404b, 404c, 404d via a b ridge r/router 405a, 406b, 
406c, 406d. A separate interface 408 is required be- 
tween a LAN and its proximate transport node for each 
other LAN with which it will exchange data packets. Be- 
tween any two LAN's that wish to exchange data pack- 
ets a separate SIS payload must be pre-assigned. 
[001 7] For example between 402a and 402d payload 
411 is assigned. If there are no packets to travel from 
LAN 402a to LAN 402d then the payload 411 will travel 
from 404a to 404d empty. 

[001 8] As shown in Figure 1 , by way of example, there 
are six channels between each transport nodes. Every 
clock period a payload 411 will be sent from one node 
to the adjacent node. Thus, the six channels are repre- 
sented by one set of payloads 411 , 412. 413. 414. 415. 
416. As shown in the figure, each payload is pre-as- 
signed to carry information between two specified trans- 
port nodes. The number of interfaces and pre-assigned 
payloads increases geometrically as additional LANS 
are associated with transport nodes on the SONET ring 
network. In contrast, as will become apparent, with a 
preferred embodiment of the present invention only one 
interface is required between a LAN and its proximate 
transport node regardless of how many LAN's on the 
system wish to exchange data packets. As well, the 
number of payloads remains constant even when more 
LANs are added. Payload travel between nodes is not 
pre-assigned, but is shared dynamically and rationally. 
[0019] Turning to Figure 2, a communications system 
100 made in accordance with this embodiment has a 
number of transport nodes 102, 104, 106, 108. Trans- 
port nodes 102, 104, 106, 108 have between them fibre 
optic pairs 110. Between some transport nodes such as 
transport node 102 and transport node 104 there may 
be an optical repeater 112. As will be obvious to those 
skilled in the art, the transport nodes 1 02, 1 04, 1 06, 1 08 
form a SONET ring network. As will also be apparent, 
more than one optical repeater could be located be- 
tween two adjacent transport nodes. 
[0020] In the preferred embodiment of the invention a 
single optical fibre is used for unidirectional transmis- 
sion of optical signals from one transport node to an ad- 
jacent transport node. Thus, for example, one optical fi- 
bre 114 in fibre optic pair 110a transmits optical signals 
from transport node 102 to transport node 104 and the 
other optical fibre 115 in fibre optic pair 110a transmits 
optical signals from transport node 104 to transport 
node 102. In other words, the SONET network is bi-di- 
rectional. 

[0021] Transport node 102 is connected to a packet 
generating device such as local area network (LAN) 116 
with a number of packet generating and receiving de- 



vices 118 connected to it In the preferred embodiment 
LAN 116 is a gigabit ethernet LAN. Alternatively, a de- 
vice could be the only device on the LAN and connected 
to a transport node, such as device 119 connected to 
5 transport node 108. As a further alternative, a transport 
node could be connected to more than one LAN. For 
example, transport node 106 is connected to LAN 120 
and LAN 126. Each device 118, 119, has a different fJie- 
dium Access Control (MAC) address. The MAC address 
70 uniquely identifies the device and is used to route data 
packets from one device to another. In overview, with 
communications system 100 ethemet data packets may 
be transmitted from a device 118a connected to LAN 
116 to transport node 102. In node 102 these packets 
IS are included in one or more SONET payloads and trans- 
mitted on a dedicated SONET channel through optical 
fibre 110a to another transport node, such as transport 
node 104 on the SONET network, servicing, via LAN 
1 1 7. a device, such as device 1 1 8b, having a destination 
20 MAC address found in the header of the ethernet data 
packets. Similarly, packets may be exchanged between 
LAN 116 and LAN 120. between LAN 117 and LAN 120 
and between any of the LANs and device 119. 
[0022] Ethemet data packets are included in SONET 
25 payloads by prefixing SONET header information and 
appending SONET trailer information to the data pack- 
ets. At the destination transport node the ethemet data 
packets are removed from one or more SONET pay- 
loads, re-assembled if they were spread out over more 
30 than one SONET payload, and then transmitted to the 
destination device. 

[0023] Initially when a given first packet generating 
device on a LAN, such as device 118a, transmits a data 
packet to a transport node, the data packet propagates 
35 around the SONET ring network in both directions. Ta- 
bles in each transport node are updated with source ad- 
dress information relating to the first device and with in- 
formation about the path in the SONET ring network the 
data packet has travelled. If another device replies to 
40 that first device, the stored source information and path 
information in the table in each transport node are used 
to route any packets back from the replying device to 
the first device. The stored source information and path 
inforrriation can also be used to help route data packets 
45 from any other device to the first device. 

[0024] Ethemet data packets can have a variable size 
or length. Thus, a packet can be spread among one or 
more SONET payloads. As well, a single SONET pay- 
load could contain parts of more than one ethernet data 
50 packet. In the preferred embodiment, the method by 
which data packets are spread among one or more 
SONET payloads or are aggregated into a single pay- 
load is accomplished by a method known as HDLC-Iike 
framing, as is described in Internet Engineering Task 
55 Force documents RFC 1662 and 1619. 

[0025] Figure 3 shows a portion of a transport node 
200 in accordance with a preferred embodiment. The 
transport node of which transport node portion 200 is a 
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part receives SONET frames from optical fibres 202, 
204 and transmits SONET frames to an output path, 
namely optical fibres 206, 208. 

[0026] Optical fibre 202 is connected to a SONET 
transport node channel add/drop 242 which removes 
SONET section and line overhead information and then 
diverts SONET channels dedicated to the transport of 
data packets to SONET packet receiver 210. The SON- 
ET transport node channel add/drop is not part of the 
present invention but is shown to illustrate the context 
of an embodiment of the invention. Those SONET chan- 
nels not dedicated to the transport of data packets are 
output on line 256 to the rest of the transport node to be 
handled in the usual manner for SONET channels. Sim- 
ilarly optical fibre 204 is connected to a SONET trans- 
port node channel add/drop 244 which removes SONET 
section and line overhead information and then diverts 
SONET channels dedicated to transport of data packets 
to SONET packet receiver 212, and diverts nondedicat- 
ed SONET channels to output line 254 which leads to 
the rest of the transport node to be handled in the usual 
manner for SONET channels. The SONET packet re- 
ceivers 210, 212 determine when each SONET frame 
within the diverted channel or channels starts and ends. 
Packet receiver 210 detects the beginning and end of 
each SONET frame and removes data packets or parts 
thereof by removing SONET header and trailer path 
overhead information leaving only the encapsulated 
packets. A data packet can be spread over a number of 
SONET payload frames. The packet receiver 210 also 
detects the start and end of each data packet. A buffer 
246 is used to store parts of a data packet received from 
different SONET payloads until the data packet is re- 
assembled in buffer 246 and can be transmitted in its 
entirety to packet distributor 214. SONET packet receiv- 
er 212 has a buffer 250 for a similar purpose. 
[0027] Transport node 200 receives ethernet data 
packets from an associated ethernet LAN through input 
line 21 6 which is connected to an ethernet receiver 218. 
Ethernet data packets are transmitted from ethernet re- 
ceiver 218 to data packet encapsulator 21 9. Preferably, 
data packets are encapsulated by appending a time-to- 
live (TTL) indicator and other information to the data 
packet. Preferably, the TTL indicator and other informa- 
tion are prefixed to the data packet. The use of the TTL 
indicator will be described below. The encapsulated da- 
ta packets are then transmitted to packet distributor 214. 
As they travel through the communications system all 
data remains encapsulated. In other words, the prefixed 
information, such as the TTL indicator, is treated as part 
of the data packet. 

[0028] The packet distributor 214 determines in a 
method described below how packets will be routed. A 
packet may be output on one or more of optical fibres 
206, 208 and output path 220 which is connected to the 
associated ethernet LAN. Packets from the packet dis- 
tributor may be loaded into a set of queues 222a. . .222n, 
associated with scheduler 228, SONET transmitter 230, 



SONET transport node channel add/drop 258 and opti- 
cal fibre 206. Alternatively, packets can be loaded into 
a second set of queues 226a... 226n associated with 
scheduler 232 and SONET transmitter 234, SONET 
5 transport node channel add/drop 262, and optical fibre 
208. As a further alternative, packets can be loaded into 
a third set of queues 224a.... 224n associated with 
scheduler 236, data packet de-encapsulator 237, ether- 
net transmitter 238 and output path 220. There can be 
any number of queues in each of the sets of queues. 
[0029] The scheduler 228, 232, or 236 selects a par- 
ticular queue from the set of queues with which it is as- 
sociated and then transmits a packet from the selected 
queue to the transmitter. In the case of schedulers 228 
and 232, the packet is transmitted to SONET transmit- 
ters 230 and 234, respectively. Assuming the packet is 
sized to fit in one SONET payload, the SONET trans- 
mitter appends SONET path overhead information to 
the packet, then transmits the packet to a SONET trans- 
port node channel add/drop 258 and 262 where SONET 
section and line overtiead information is appended to 
the packet. The packet is then included in a SONET pay- 
load inserted into a SONET channel dedicated for data 
packet transport and recombined with channels not ded- 
icated to data packet transport arriving on lines 260 and 
264. The combined channels are output on an optica! 
fibre 206 or 208. If the packet must be spread over a 
number of SONET payload frames, the packet is loaded 
into a buffer, such as buffer 248 or buffer 252, when the 
packet is received by SONET transmitters 230 and 234. 
Parts of each packet are removed from the buffer, have 
SONET overhead information appended and are includ- 
ed in a SONET payload and sent to the SONET trans- 
port node channel add/drop 258 and 262. This is repeat- 
ed until the entire data packet has been transmitted. If 
a packet or end portion of a packet is smaller than a 
SONET payload, there will be a remaining unused por- 
tion of the SONET payload. In this situation a second 
data packet is selected from a queue associated with 
that output path and stored in the buffer. The part of the 
second data packet that will fit in the remaining portion 
of the SONET payload has SONET overhead informa- 
tion appended and is then included in the SONET pay- 
load and sent to the SONET transport node channel 
add/drop. One or more SONET other payloads will in- 
clude any remaining portions of the second data packet. 
In the case of scheduler 236, the packet is transmitted 
to data packet de-encapsulator 237 which removes in- 
formation added by the data packet encapsulator, such 
as a TTL indicator, and then transmits the packet to 
transmitter 238 which transmits the packet to the asso- 
ciated output path 220 to a LAN. 
[0030] Data packets are intercepted and re-queued at 
each transport node. This differs from conventional 
SONET transmission in that in conventional SONET an 
STS channel (comprising periodic payloads) is usually 
dedicated to traffic between two pre-determined trans- 
port nodes. In the preferred embodiment an STS chan- 
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nel(or optionally multiple channels or a part of a channel) 
is dedicated only to ethernet data packets. Those eth- 
ernet data packets can be received from or transmitted 
to any transport node on the SONET network. 
[0031] Transmission to optical fibre 206 or reception 
from optical fibre 202 is commonly referred to transmit- 
ting to or receiving from the west. Similarly, transmission 
to optical fibre 208 or reception from optical fibre 204 is 
commonly referred to as transmitting to or receiving 
from the east. The east and west directions are part of 
the SONET ring network. Similarly, transmission to or 
reception from a line not connected to the SONET ring 
network, such as output line 220 or input line 2 1 6 is com- 
monly referred to as transmitting to or receiving from the 
north. 

[0032] The manner in which data packets are routed 
from one device to another device will now be described 
in more detail. 

[0033] An ethernet data packet contains a header 
which has both a source MAC address and a destination 
MAC address. The source MAC address is the address 
of the device transmitting the ethernet data packet. In a 
preferred embodiment of the present invention, the 
source MAC address is used to assist with efficient rout- 
ing of encapsulated ethernet data packets on the SON- 
ET network. With reference to Figure 3, this is accom- 
plished in the following way: 

Step 1 

When an ethernet data packet is received, it is sent 
to the packet distributor 214, along with information 
about whether the packet was received from optical 
fibre 202, 204, or input line 216. In other words, 
whether the packet was received from the west, 
east or north. 

Step 2 

Packet distributor 214 places in an associated table 
240 the source MAC address and an identifier of 
the input source, namely optical fibre 202, 204 or 
input line 216. The identifier could be expressed as 
the direction north, west or east. 

Step 3 

Packet distributor 214 reads the destination MAC 
address of the ethernet data packet. The destina- 
tion MAC address could be read directly from the 
ethernet packet header. Alternatively, where data 
packet encapsulator 219 is configured to read the 
ethernet packet header and prefix the source and 
destination MAC addresses to the packet, the pack- 
et distributor may obtain the destination MAC ad- 
dress from this attemate source. Packet distributor 
214 refers to table 240 to determine it there is a table 
entry for a source MAC address in the table identical 
to the destination MAC address of the ethernet data 
packet. If there is a matching source MAC address, 
the packet distributor 214 reads the identifier of the 



input source corresponding to the source MAC ad- 
dress. Each input source has an associated output 
source. For example the output source associated 
with input optical fibre 202, or the west direction, is 
5 output optical fibre 206. Similarly the output source 
associated with input path 216, or the north direc- 
tion, is output path 220. 

After the packet distributor 214 reads the identifier 
of the input source it will assign the received ether- 
10 net data packet to one of the set of queues leading 
to the output source associated with the input 
source identifier of the matching source MAC ad- 
dress. 

15 [0034] A benefit of the routing of data packets dis- 
closed by this embodiment is that it allows packets to 
flow between nodes even when there was no pre-as- 
signed payload between the two nodes. 
[0035] It is possible that there will be no table entry 
20 with a source MAC address identical to the destination 
MAC address of the ethernet data packet. If this hap- 
pens, the original and a copy of the ethernet data packet 
will be sent to all output paths connected to the transport 
node, except that no copy is transmitted to the output 
25 path associated with the input path from which the eth- 
ernet data packet was received by the transport node. 
Thus, for example, if an ethernet data packet arrives 
from optical fibre 202, and there is no relevant entry in 
table 240, the original packet will be queued for trans- 
30 mission on output line 220 and a copy will be created 
and queued for transmission on optical fibre 208. Simi- 
larly, it an ethernet data packet arrives from input line 
21 6 and there is no relevant entry in table 240, the orig- 
inal packet is queued for distribution on optical fibre 206 
35 and a first copy of the packet is created and queued for 
transmission on optical fibre 208. 
[0036] In a preferred embodiment an STS channel or 
group of channels or subset of a channel is set aside to 
transmit ethernet data packets. At each transport node 
40 200, data packets are removed from the SONET pay- 
loads received from the set-aside channel and the eth- 
ernet data packets go into the packet distributor 214. It 
an ethernet data packet has a destination MAC address 
belonging to a device in the north direction, (that is, con- 
45 nected to an ethernet LAN that is connected to the trans- 
port node 200), the ethemet data packet will be queued 
up for transmission on the output line 220. The set-aside 
STS channel which has released or dropped off its SON- 
ET pay loads will then have capacity to accept a new en- 
50 capsulated packet. The new packet will be taken from 
one of the queues 222a.. .222n; 226a ..226n associated 
with SONET transmitters 230, 234. In the preferred em- 
bodiment, each queue in a set of queues corresponds 
to packets from a unique source MAC address. In these 
55 sets of queues the letter "n", such as in "222n", repre- 
sents any arbitrary number of queues. These queues 
could contain ethemet data packets that were transmit- 
ted from input line 216 to transport node 200. These 
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queues also contain ethemet data packets that are not 
destined for output line 220 but were received from other 
transport nodes via the SONET network. The set-aside 
STS channel will continuously transmit data packets 
from a transport node to its adjacent transport node, if s 
any data packets are queued for transmission. Because 
packets are re-queued at each transport node, conten- 
tion is easily managed by schedulers 228, 232. 236. 
Moreover, contention can be managed even though 
LAN devices are spread out over large distances. io 
[0037] In the preferred embodiment, a time to live 
(TTL) indicator, set at a pre-determined number, is pre- 
fixed to each ethernet data packet in the data packet 
encapsulator 21 9. The prefixed TTL indicator will be re- 
moved by data packet de-encapsuiator 237 before an 
ethernet data packet is transmitted to an ethemet LAN 
by ethemet transmitter 238. The TTL indicator is decre- 
mented every time the ethemet data packet is de-en- 
capsutated and re-queued at a transport node. 
[0038] When the TTL indicator reaches zero, the next 20 
transport node that receives it does not re-queue the 
ethernet data packet but discards It. A TTL indicator is 
typically set at a pre-determined number so that the en- 
capsulated ethernet data packet must make at least one 
complete circuit of the SONET ring network before it is ^5 
discarded. An encapsulated ethernet data packet will 
make a complete circuit if a destination f^AC address is 
specified that does not exist or is disabled, or is not 
known to exist. 

[0039] A TTL indicator of a packet can also be used 30 
to assist a transport node attached to a SONET ring to 
find the shortest path to the destination MAC address 
for the packet from that transport node. Referring to Fig- 
ure 4, the shortest path is found as follows: 

35 

Step 1 

A device 300 transmits an ethemet data packet to 
transport node 302 of SONET ring 350. A TTL indi- 
cator is set at a pre-determined value and prefixed 
to the ethernet data packet. A packet distributor 304 40 
in transport node 302 checks an associated table 
306 to see if the destination MAC address appears 
on table 306. If it does not, the original ethernet data 
packet is queued up in a queue associated with op- 
tic fibre pair 308. Also, a first copy of the ethernet 45 
data packet is created and queued up in a queue 
associated with optic fibre pair 310 for transmission 
to transport node 318. Preferably, a broadcast bit is 
set in the SONET overhead header for each pay- 
load holding the packet. so 
The original ethernet data packet is transmitted to 
transport node 312. In transport node 312, packet 
distributor 314 checks associated table 316 for an 
entry corresponding to the destination MAC ad- 
dress. If it is not found, the original ethernet data 55 
packet is re-queued for transmission to transport 
node 322. A second copy of the ethernet data pack- 
et will be created and queued for transmission to 



ethernet LAN 31 3. Further at packet distributor 314 
of transport node 31 2, an entry is made in an asso- 
ciated table 316 of (i) the source MAC address of 
device 300, (ii) an identifier of the input source or 
direction, namely optic fibre 308. and (iii) the value 
of the TTL indicator in table 316 associated with the 
packet distributor 314. This process continues at 
each transport node such that when (the original or 
a copy of) the packet reaches each transport node, 
the packet distributors 314, 328, 326, 324 associat- 
ed with each transport node checks if the source 
MAC address is in its associated tables 316, 334, 
332, 330. If it is not. (i) the source MAC address of 
device 300, (ii) identifier of the input source, and (iii) 
TTL value are stored in its tables 31 6, 334, 332. 330 
and extra copies of the ethernet data packet are cre- 
ated for transmission to any LANs or other output 
paths associated with those transport nodes. At 
each transport node, the TTL associated with the 
ethernet data packet is decremented. 
Meanwhile, the first copy of the ethemet data pack- 
et travels around the SONET ring in the opposite 
direction. Thus, each transport node will receive 
both the original ethernet data packet and the first 
copy ethernet data packet. Because the original 
ethernet data packet and first copy ethernet data 
packet travel in opposite directions, either packet 
could be received first by a transport node, depend- 
ing on the location of the transport node. The first 
time a packet from a given source is received by a 
node, it results in the creation of an entry in the 
transport nodes' associated table. Subsequent 
packets received from this source are compared 
with this entry. Thus, if the source MAC address is 
in the associated table, the value of the TTL indica- 
tor of the stored value and the value of the TTL in- 
dicator associated with the subsequently received 
packet are compared. A higher value of TTL indica- 
tor indicates tewer transport nodes between the 
source MAC address and that transport node. The 
associated table saves the source MAC address, 
identifier of the input source or direction and TTL 
value associated with the higher TTL value and dis- 
cards the source MAC address, identifier of the in- 
put source and TTL value associated with the lower 
TTL value. Then as described above, subsequent 
packets with a destination MAC address equal to 
the source MAC address stored on the associated 
table are routed to the output path associated with 
the input source identifier. In this way, the subse- 
quent packets are routed in the direction which will 
have the fewest intervening transport nodes. 
The original packet and the first copy packet are not 
re-queued when they reach the transport node 302 
from which they were first transmitted. Alternatively, 
they are discarded when their TTL has decrement- 
ed to zero. In a preferred embodiment, the entries 
on the associated tables are times tamped and de- 
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leted alter a fixed amount of time, such as is spec- 
ified in IEEE standard 802. ID. 
Although the description has referred to SONET 
rings and SONET network rings, the invention ap- 
plies generally to SONET networks, such as a SON- 
ET mesh network. The exception to this is finding 
the shortest path, which requires a bi-directional 
ring. The described method of using aTTL indicator 
to find the shortest path to a given address is not 
restricted to SONET rings. It is applicable to any 
system sending packetized data having source and 
destination addresses in the packet over a bidirec- 
tional loop network. 

Occasionally a path between two transport nodes 
will fail. For example, it may be impossible to trans- 
mit packets from transport node 302 to transport 
node 312 through optical fibre pair 308. This failure 
is detected by an operator or by operations and 
maintenance devices. When this happens, all en- 
tries in table 306 where the input source identifier 
is associated with optical fibre 308 are deleted. This 
ensures that subsequent packets will not be queued 
in this direction. As well, packets are removed from 
the queue for output optical fibre 308 and trans- 
ferred to another queue. 

[0040] Numerous modifications, variations and adap- 
tations may be made to the particular embodiments of 
the invention described above without departing from 
the scope of the invention which is defined in the claims. 30 
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Claims 



2. 



A method tor routing data packets in a SONET net- 
work comprising the steps of: 

(a) receiving a data packet at a transport node ; 

(b) determining a destination address for said 
data packet; 

(c) selecting an output path based on said des- 
tination address; 

(d) queuing said data packet on a queue asso- 
ciated with said selected output path; 

(e) selecting a data packet from said queue; 
and 

(f) transmitting said selected data packet onto 
said selected output path. 

A method as claimed in claim 1 further comprising 
the steps of, for each data packet: 

(a) determining whether a time-to-live (TTL) in- 
dicator has been appended to said each data 
packet; 

(b) if a TTL indicator having a value has not al- 
ready been appended, appending a TTL indi- 
cator having a value to said each data packet; 
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and 

(c) if a TTL indicator having a value has already 
been appended to said each data packet, dec- 
rementing the value of said TTL indicator 

3. A method as claimed in claim 2 wherein said SON- 
ET network is a SONET ring and further comprising 
the steps of: 

(a) for each data packet: 

(i) associating said each data packet with 
an identifier of an input path from which 
said each data packet was received; 

(ii) determining if an entry exists in a table 
associated with said transport node match- 
ing a source address of said each data 
packet and. if said entry does not exist, cre- 
ating an entry on an associated table con- 
taining (i) a source address of said each 
data packet, (ii) said identifier and (iii) the 
value of the TTL indicator; and 

(b) for subsequent data packets: 

comparing a source address of said sub- 
sequent data packet with the source ad- 
dress entries on said table and, on finding 
a matching address on said table, creating 
a new table entry based on said subse- 
quent data packet and deleting an existing 
matching entry if a value of the TTL indica- 
tor is higher in said subsequent data packet 
than that in said existing entry. 

4. A method as claimed in claim 2 or claim 3 further 
comprising the step of: 

(a) dropping a data packet from a transport 
node if its TTL indicator has decremented to ze- 
ro. 

5. A method as claimed in any of the above claims 
wherein said selected output path is on the SONET 
network and further comprising the step of: 

(a) including said selected data packets in one 
or more SONET payloads prior to transmitting 
said data packets onto said selected output 
path. 

6. A method as claimed in any of the above claims, 
wherein the step of selecting an output path based 
on said destination address comprises the following 
sub-steps: 

(i) determining if an entry exists in a table as- 
sociated with said transport node matching said 
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destination address; and, 
(ii) ff a matching entry exists, reading from said 
associated table an identifier of an output path 
associated with said destination address, said 
output path of said identifier comprising said s 
selected output path. 

A method as claimed in claim 6 further comprising 
the steps of: 

10 

(a) receiving an indication an output path is not 
functional; and 

(b) deleting any entries in said table containing 
an identifier of said non-functional output path. 

IS 

A method claimed in any of the above claims where- 
in the step of selecting an output path associated 
with said destination address comprises the follow- 
ing sub-steps: 

20 

(i) determining if an entry exists in a table as- 
sociated with said transport node matching said 
destination address, and 

(ii) if a matching entry does not exist: 

2S 

(I) selecting an output path for said each 
data packet; 

(II) for at least one remaining output path 
other than said selected output path creat- 
ing a copy of said each data packet; 30 

(III) associating each copy of said each da- 
ta packet with one of said remaining output 
paths; 

(IV) queuing said copy data packets on 
queues of said associated remaining out- 3S 
put paths; 

(V) selecting copy data packets from said 
queues of remaining output paths; and 

(VI) transmitting said selected copy data 
packets onto said associated remaining 40 
output paths. 

An apparatus for routing data packets in a SONET 
network comprising: 

45 

(a) means for receiving a data packet at a trans- 
port node: 

(b) means for determining a destination ad- 
dress for said data packet; 

(c) means tor selecting an output path based so 
on said destination address; 

(d) means for queuing said data packet on a 
queue associated with said selected output 
path; 

(e) means for selecting a data packet from said 
queue; and 

(f) means for transmitting said selected data 
packet onto said selected output path. 



10. An apparatus for routing data packets in a SONET 
network comprising: 

(a) a pluralrty of SONET packet receivers for 
receiving SONET payloads, each SONET pay- 
load containing at least part of a data packet; 

(b) a plurality of data packet receivers for re- 
ceiving data packets; 

(c) a packet distributor, 

(i) said packet distributor connected to 
each of said plurality of SONET packet re- 
ceivers and to each of said plurality of data 
packet receivers; and 

(ii) said packet distributor for receiving data 
packets from said plurality of SONET pack- 
et receivers and data packet receivers; 

(d) a plurality of output paths; 

(e) means for routing each data packet to a se- 
lected output path, based on a destination ad- 
dress of said each data packet; 

(f) a set of queues associated with each said 
output path, said set of queues for receiving 
routed data packets; 

(g) a scheduler associated with each of said set 
of queues for selecting a data packet from one 
of said set of queues; 

(h) a transmitter for transmitting said selected 
data packet to an output path. 

11 . A method for selecting a shortest path to a destina- 
tion address for data packets containing source and 
destination addresses in a bi-directional ring net- 
work comprising the steps: 

(a) receiving said data packets at a node; 

(b) for each received data packet; 

(ii) appending a TTL indicator having a val- 
ue to said each data packet if a TTL indi- 
cator has not already been appended; 

(iii) decrementing a value of a TTL indicator 
if a TTL indicator has 

(iv) already been appended to said each 
data packet 

(iii) compahng a source address of said 
each data packet with source address en- 
tries on said table and on not finding a 
match, 

(I) associating said each data packet 
with an identifier of an output path as- 
sociated with the input path from which 
said data packet was received; 

(II) creating an entry on a table asso- 
ciated with said node containing: 
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(A) a source address of said each 
data packet, (B) said identifier, 
and (C) the value of the TTL indi- 
cator; 

and on finding a nnatch creating a 5 
new table entry based on said 
each data packet and deleting an 
existing matching entry if a value 
of the TTL indicator is higher in 
said each data packet than that in io 
said existing entry. 

(c) selecting an output path for a given data 
packet by: 

15 

(i) determining if an entry exists in said 
associated table matching a destina- 
tion address for said given data pack- 
et, 

(ii) if said entry exists, reading from 20 
said associated table an identifier of an 
output path associated with said des- 
tination address, said read output path 
comprising said selected output path; 

(iii) if said entry does not exist, 

(A) selecting an output path for 
said given data packet; 

(B) creating at least one copy of 
said given data packet; 

(C) selecting a remaining output 
path other than said selected out- 
put path for each copy of said giv- 
en data packet. 

35 

1 2. A method as claimed in claim 1 1 , including the steps 
of: 

(a) queuing said given data packet on a queue 
associated with said selected output path and 40 
each copy of said data packet, if any, on a 
queue associated with a remaining output path; 

(b) selecting data packets from said queue; and 

(c) transmitting said selected data packets onto 
said each output path. 

1 3. An apparatus for selecting a shortest path to a des- 
tination address for data packets containing source 
and destination addresses in a bi-directional ring 
network comprising: 

(a) means for receiving said data packets at a 
node; 

(b) means for, tor each received data packet; 

55 

(i) appending a TTL indicator having a val- 
ue to said each data packet if a TTL indi- 
cator has not already been appended; 



(ii) decrementing a value of a TTL indicator 
if a TTL indicator has already been ap- 
pended to said each data packet: 

(iii) comparing a source address of said 
each data packet with source address en- 
tries on said table and on not finding a 
match, 

(I) associating said each data packet with 
an identifier of an output path associated 
with the input path from which said data 
packet was received; 

(II) creating an entry on a table associated 
with said node containing: 

(A) a source address of said each data 
packet, (B) said identifier, and (C) the 
value of the TTL indicator; 
and on finding a match creating a new 
table entry based on said each data 
packet and deleting an existing match- 
ing entry if a value of the TTL indicator 
is higher in said each data packet than 
that in said existing entry. 

(c) means for selecting an output path for a giv- 
en data packet by: 

(i) determining if an entry exists in said as- 
sociated table matching a destination ad- 
dress for said given data packet, 

(ii) if said entry exists, reading from said as- 
sociated table an identifier of an output 
path associated with said destination ad- 
dress, said read output path comprising 
said selected output path; 

(iii) if said entry does not exist, 

(A) selecting an output path for said 
given data packet; 

(B) creating at least one copy of said 
given data packet; 

(C) selecting a remaining output path 
other than said selected output path 
for each copy of said given data pack- 
et. 

14. An apparatus as claimed in claim 13, further com- 
prising: 

(a) means for queuing said given data packet 
on a queue associated with said selected out- 
put path and each copy of said data packet, if 
any. on a queue associated with a remaining 
output path; 

(b) means for selecting data packets from said 
queue; and 

(b) means for transmitting said selected data 
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